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We  started  the  construction  of  a  system-level  simulation  package  incorporating  the  ideas  of 
temporal  tagging  for  attentional  moduladon  of  neuronal  signals  into  a  model  of  several 
interacting  cortical  and  subcortical  areas  using  video-camera  acquired  images.  This  simulation 
differs  significantly  from  connection  ist  models,  since  the  temporal  structure  of  the  neural  signals 
plays  a  crucial  role  in  the  coding  of  attentional  state.  Thus,  we  use  single  neuron  models  with 
quasi  realistic  temporal  behavior.  In  particular,  we  exclude  all  models  in  which  the  activity  of  a 
neuron  is  measured  only  by  average  firing  rate.  Instead,  we  assume  a  spiking  mechanism,  and 
that  the  communication  between  neurons  has  to  be  accomplished  by  the  exchange  of  potentials. 
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Progress  Report 

October,  1. 1992  -  September  30, 1993 

Our  research  has  had  progress  on  two  different  fronts. 

L  Design  of  Simulated  Primate  Visual  System 

We  started  the  construction  of  a  system-level  simulation  package  incorporating  the 
ideas  of  temporal  tagging  for  attentional  modulation  of  neuronal  signals  into  a 
model  of  several  interacting  cortical  and  subcortical  areas  using  video-camera 
acquired  images.  This  simulation  differs  significantly  from  connectionist  models, 
since  the  temporal  structure  of  the  neural  signals  plays  a  crucial  role  for  the  coding 
of  attentional  state.  Thus,  we  use  single  neurons  models  with  quasi  realistic 
temporal  behavior.  In  particular,  we  exdude  all  models  in  which  the  activity  of  a 
neuron  is  measured  only  by  average  firing  rate.  Instead,  we  assume  a  spiking 
mechanism,  and  that  the  communication  between  neurons  has  to  be  accomplished 
by  the  exchange  of  action  potentials. 

The  systems-level  approach  by  nature  entails  that  the  number  of  neurons  to  be 
simulated  will  be  relatively  large.  In  the  interest  of  obtaining  reasonable 
performance  with  the  available  hardware,  and  also  in  order  to  control  ^e  number  of 
free  parameters  of  the  .system,  we  have  had  to  limit  the  level  of  biological  realism  of 
the  single  cell  model.  Therefore,  we  dedded  against  a  full  compartmental  modehng 
scheme,  on  the  assumption  that  the  details  of  the  dendritic  arborization  are  not 
essential  to  the  behavior  of  the  system  as  a  whole.  On  the  other  hand,  we  have 
devoted  considerable  effort  to  develop  a  realistic  model  of  the  temporal  structure  of 
the  neural  response  which  goes  beyond  that  of  a  simple  integrate-and-fire  neuron. 
For  instance,  after  a  spike  has  been  generated,  we  simulate  the  opening  of 
potassium  channels  which  lead  to  the  repolarization  of  the  membrane  voltage. 

Our  evolving  simulation  package  is  a  tool  which  we  intend  to  apply  to  different 
single  neuron  models  and  different  anatomical  structures.  Therefore,  for  the  sake  of 
flexibility,  the  simulation  program  was  developed  with  the  goal  of  allowing  a 
maximum  of  adaptability.  We  used  the  object-oriented  language  which  allows 
the  definition  of  object  types  (called  "classes"  in  with  an  inheritance  structure. 
For  instance,  the  simixlator  employs  a  number  of  different  neuron  classes,  all  of 
which  are  derived  from  a  base  neuron  class  which  specifies  general  neuron 
characteristics  (e.g.  they  generate  spike  trains,  they  receive  input  via  synaptic 
objects,  they  have  connections  ••  with  weights  and  propagation  delays  —  to  synapses 
on  other  neurons,  etc.).  However,  different  derived  neuron  classes  may  implement 
these  basic  operations  in  different  ways,  and  they  may  have  additional  functionality 
not  possessed  by  the  base  class.  Some  classes  of  neurons^  for  instance,  generate 
spikes  according  to  a  Poisson  process  whose  time-dependent  average  rate  depeni^ 
on  stimulus  parameters  and/or  attentional  state.  Other  classes  integrate  their 
synaptic  input,  which  may  include  stochastic  (noise)  contributions,  and  generate 
spikes  deterministically  based  only  on  this  input. 
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The  input  of  the  simulator  consists  of  natural  images  recorded  with  a  video  camera, 
digitized  and  entered  into  computer  memory.  The  early  stages  of  visual  processing 
stages,  which  in  biological  terms  correspond  to  the  retina,  the  lateral  geniculate 
nucleus  (LGN)  and  layer  4  of  cortical  area  VI,  are  collectively  simulated  as  a 
population  of  orientation  selective  Gabor  filters  at  different  spatial  wavelengths. 
For  the  sake  of  performance,  we  do  not  simulate  individual  retinal  and  LGN  cells 
(as  we  did  in  our  previous  work;  see  Worgotter  and  Koch,  1991,  Worgbtter  et  al, 
1991).  The  Gabor  filter  outputs  are  used  as  the  input  for  orientation-selective  VI 
cells  outside  layer  4.  This  input  determines  the  mean  spike  rates  of  the  VI  cells. 
The  model  used  for  these  cells  was  of  the  type  described  above,  i.e.  they  were 
modeled  as  Poisson  processes  with  average  rate  determined  by  their  input.  For  now, 
only  two  orientations  (horizontal  and  vertical)  have  been  used  for  testing;  a  larger 
variety  of  such  filters  will  be  added  later. 

The  access  to  the  simulator  is  via  a  graphical  user  interface.  In  the  enclosed  figure, 
an  example  of  the  activity  in  some  of  the  simulated  neural  populations  is  shown, 
together  with  the  camera  image  used  to  generate  this  activity.  The  image,  shown  at 
the  center  of  the  figure,  shows  a  part  of  a  book  shelf.  On  the  top  are  shown  the 
activities  of  two  populations  which'  correspond  to  the  above  mentioned  Gabor  filters 
of  a  given  spatial  wave  length  (here  3  pixels/cycle).  The  population  on  the  left 
consists  of  cells  sensitive  to  vertical  input,  the  population  to  the  right  consists  of 
cells  sensitive  to  horizontal  input.  The  cell  activity,  in  this  case  corresponding  to  the 
average  firing  rate  of  a  Poisson  process,  is  coded  on  a  color  scale  where  cooler  colors 
correspond  to  lower  activity.  When  the  Poisson  process  of  a  specific  cell  generates  an 
event,  the  color  of  this  cell  is  set  to  red  and  the  cell  sends  a  spike  to  its  postsynaptic 
partners.  Note  the  differences  in  activity  due  to  the  different  orientations  of  edges 
in  the  original  image.  Such  differences  are  also  visible  in  the  next  stage,  shown  in 
the  two  lower  images.  They  show  the  activity  in  the  neural  populations  to  which  the 
Gabor  filter  cells  represented  above  them  project.  In  this  case,  the  color  shows  the 
simulated  intracellular  voltage  of  the  respective  cell,  lower  (more  negative)  values 
being  represented  by  cooler  colors.  Again,  action  potentials  are  shown  as  red  pixels. 

We  are  currently  working  toward  the  integration  into  our  system  of  the  V4 
representation  and  the  saliency  map,  so  that  we  may  begin  to  attack  the  central 
question  of  this  work  regarding  the  possible  biophysical/neuronal  basis  for  selective 
visual  attention. 

IL  Temporal  Tagging  in  .Single  Cortical  Cells 

Let  me  remind  you  of  the  k«*v  ideas.  In  1990,  Francis  Crick  and  I  proposed  that  the 
action  of  attention  at  the  neuronal  level  is  two-fold.  Firstly,  attention  tags  or  labels 
the  output  of  individual  neurons  (in  that  sense,  an  "attended  signal"  carries  the 
same  information  as  an  un  attended  signal",  plus  an  additional  label).  The  second 
action  of  attention  is  to  activate  competition  among  neurons.  In  the  un-attended 
state,  nearby  objects  in  visual  space  excite  different  but  neighbouring  neurons  in 
cortex.  Directing  one's  attention  to  one  of  the  objects  will  "label"  the  neurons  in  VI 
responding  to  this  object  and  will  also  activate  competition  in  V4  and  IT  among 
neurons  responding  to  the  two  objects.  In  this  competition,  the  ’labeled"  or  "tagged" 


object  will  now  win  out,  suppressing  the  response  of  the  neuron  to  the  non-attended 
object.  This  theory?^cplains  the  seminal  experiment  of  Moran  and  Desimone  on  the 
effect  of  attention  in  V4  or  IT  neurons. 

How  should  this  tagging  be  implemented  at  the  level  of  neurons?  Furthermore,  how 
can  this  "tagging"  or  "labeling"  be  de-modulated  at  the  level  of  V4  or  IT.  We 
suggested  two  possibilities,  compatible  with  our  detailed  knowledge  of  the 
biophysics  of  cortical  cells.  One  mechanism  assumes  that  the  action  of  attention  at 
the  level  of  VI  is  to  impart  a  small  oscillatory  modulation  (in  the  40  Hz 
neighborhood)  to  neurons  responding  to  objects  within  the  "attentional  searchlight." 
Neurons  in  primary  visual  cortex  respond  to  visual  stimuli  with  a  Poisson 
distributed  spike  train  with  an  appropriate,  stimulus-dependent  mean  firing  rate. 
The  firing  rate  of  neturons  whose  receptive  fields  overlap  with  the  "focus  of 
attention"  is  modidated  with  a  periodic  fiinction  in  the  40  Hz  range,  such  that  their 
mean  firing  rate  is  identical  to  the  mean  firing  rate  of  neurons  in  "non-attended" 
areas.  This  modulation  is  detected  by  inhibitory  intemeurons  (interneurons 
selectively  responding  to  frequency-modulated  inputs  have  been  described  by  R. 
Llinas  and  co-workers  in  1991)  in  V4  and  is  used  to  suppress  the  response  of  V4 
cells  associated  with  non-attended  visual  stimuli.  Using  very  simple  single-cell 
models,  we  obtain  quantitative  agreement  with  Moran  and  Desimone's  (1985) 
experiments  (coming  out  in  Vision  Research  ). 

A  second  mechanism  we  investigated  assumes  that  the  cross-correlation  among 
neurons  in  VI  is  modulated  by  attention.  While  neurons  responding  to  non-attended 
objects  are  assumed  to  fire  in  an  independent  Poisson  maimer,  neurons  responding 
to  an  "attended  object "  fire  in  a  temporally  synchronized  manner;  in  other  words, 
the  cross-correlation  among  them  is  increased.  Because  the  firing  pattern  of 
individual  neurons  is  random  with  the  same  mean  rates  in  both  the  attended  and 
the  non-attended  cases,  it  will  not  be  possible  to  pick  up  this  subtle  "tagging"  using 
single-unit  recordings.  Intemeurons  in  V4  respond  stronger  to  the  synclmonized 
than  to  the  de-synchronized  input  from  VI  and  inhibit  the  response  to  the  non- 
attended  object. 
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